AVATecH ― automated annotation through audio and video analysis

In different fields of the humanities annotations of multimodal resources are a necessary component of the research workflow. Examples include linguistics, psychology, anthropology, etc. However, creation of those annotations is a very laborious task, which can take 50 to 100 times the length of the annotated media, or more. This can be significantly improved by applying innovative audio and video processing algorithms, which analyze the recordings and provide automated annotations. This is the aim of the AVATecH project, which is a collaboration of the Max Planck Institute for Psycholinguistics (MPI) and the Fraunhofer institutes HHI and IAIS. In this paper we present a set of results of automated annotation together with an evaluation of their quality

Application of video processing methods for linguistic research

Evolution and changes of all modern languages is a well-known fact. However, recently it is reaching dynamics never seen before, which results in loss of the vast amount of information encoded in every language. In order to preserve such heritage, properly annotated recordings of world languages are necessary. Since creating those annotations is a very laborious task, reaching times 100 longer than the length of the annotated media, innovative video processing algorithms are needed, in order to improve the efficiency and quality of annotation process

A methodology for user Interface adaptation of multi-device broadcast-broadband services

New audiovisual experiences involve consuming several contents displayed through multiple internet-connected devices. The TV is still the central hub of the living room, but it is often used simultaneously with other screens. Consequently, the user has the chance to consume all different contents at once across multiple devices. However, no existing adaptation models are available to dynamically adapt such a multitude of contents in multi-device contexts. To address this gap, this paper proposes a novel multi-device adaptation methodology to build adaptive User Interfaces for multi-screen hybrid broadcastbroadband TV experiences. The methodology is extensible to any kind of content, device and user, and is applicable to different contexts considering technological evolution and other fields of application. The proposed methodology is the outcome of extensive research that arose from a previous multi-device media service deployment with broadcasters

Inter-Modal Selective 3D Coding of PET-CT Datasets

In this work we introduce a new selective coding approach suitable for co-registered multi-modal medical images and we apply it to large PET-CT volumes. Salience information guiding a space variant reconstruction quality of the anatomical volume (CT) is generated through an automatic analysis of the functional volume (PET). This allows a versatile multiple volume-of-interest coding with arbitrary 3D-shape and scaling-factors and without the need of side information to be transmitted. The proposed solutions are suitable for critical applications where high and optimized compression ratio, minimization of human intervention and full diagnostic quality preservation are all required

Application of audio and video processing methods for language research

Annotations of media recordings are the grounds for linguistic research. Since creating those annotations is a very laborious task, reaching 100 times longer than the length of the annotated media, innovative audio and video processing algorithms are needed, in order to improve the efficiency and quality of annotation process. The AVATecH project, started by the Max-Planck Institute for Psycholinguistics (MPI) and the Fraunhofer institutes HHI and IAIS, aims at significantly speeding up the process of creating annotations of audio-visual data for humanities research. In order for this to be achieved a range of state-of-the-art audio and video pattern recognition algorithms have been developed and integrated into widely used ELAN annotation tool. To address the problem of heterogeneous annotation tasks and recordings we provide modular components extended by adaptation and feedback mechanisms to achieve competitive annotation quality within significantly less annotation time

Generation of induced Pluripotent Stem Cells (UNIBSi008-A, UNIBSi008-B, UNIBSi008-C) from an Ataxia-Telangiectasia (AT) patient carrying a novel homozygous deletion in ATM gene.

Abstract Using a Sendai Virus based vector delivering Yamanaka Factors, we generated induced Pluripotent Stem Cells (iPSCs) from peripheral blood mononuclear cells of a patient affected by Ataxia Telangiectasia (AT), caused by a novel homozygous deletion in ATM, spanning exons 5 to 7. Three clones were fully characterized for pluripotency and capability to differentiate. These clones preserved the causative mutation of parental cells and genomic stability over time (>100 passages). Furthermore, in AT derived iPSCs we confirmed the impaired DNA damage response after ionizing radiation. All these data underline potential usefulness of our clones as in vitro AT disease model

Generation of three iPSC lines from fibroblasts of a patient with Aicardi Goutières Syndrome mutated in TREX1

Fibroblasts from a patient with Aicardi Gouti\ue8res Syndrome (AGS) carrying a compound heterozygous mutation in TREX1, were reprogrammed into induced pluripotent stem cells (iPSCs) to establish isogenic clonal stem cell lines: UNIBSi006-A, UNIBSi006-B, and UNIBSi006-C. Cells were transduced using the episomal Sendai viral vectors, containing human OCT4, SOX2, c-MYC and KLF4 transcription factors. The transgene-free iPSC lines showed normal karyotype, expressed pluripotent markers and displayed in vitro differentiation potential toward cells of the three embryonic germ layers

A novel primary human immunodeficiency due to deficiency in the WASP-interacting protein WIP

A female offspring of consanguineous parents, showed features of Wiskott-Aldrich syndrome (WAS), including recurrent infections, eczema, thrombocytopenia, defective T cell proliferation and chemotaxis, and impaired natural killer cell function. Cells from this patient had undetectable WAS protein (WASP), but normal WAS sequence and messenger RNA levels. WASP interacting protein (WIP), which stabilizes WASP, was also undetectable. A homozygous c.1301C>G stop codon mutation was found in the WIPF1 gene, which encodes WIP. Introduction of WIP into the patient’s T cells restored WASP expression. These findings indicate that WIP deficiency should be suspected in patients with features of WAS in whom WAS sequence and mRNA levels are normal

The co-creation space: Supporting asynchronous artistic co-creation dynamics

Artistic co-creation empowers communities to shape their narratives, however HCI research does not support this multifaceted discussion and reflection process. In the context of community opera, we consider how to support co-creation through the design, implementation, and initial evaluation of the Co-Creation Space (CCS) to help community artists 1) generate raw artistic ideas, and 2) discuss and reflect on the shared meaning of those ideas. This work describes our user-centered process to gather requirements and design the tool, and validates its' usability with 6 community opera participants. Our findings support the value of our tool for group discussion and personal reflection during the creative process

A novel production workflow and toolset for opera co-creation towards enhanced societal inclusion of people

Opera uses all the visual and performing arts to create extraordinary worlds of passion and sensibility. It is rightly recognised as a great achievement of European culture. And yet a form that once inspired social and artistic revolutions is often seen as the staid preserve of the elite. With rising inequality and social exclusion, many see opera—if they think of it at all—as symbolic of what is wrong in Europe today. This paper presents the technological and scientific approach of the European H2020 TRACTION project that aims to use opera as a path for social and cultural inclusion, making it once again a force for radical transformation. TRACTION wants to define new forms of artistic creation through which the most marginalised groups (e.g. migrants, the rural poor, young offenders and others) can work with artists to tell the stories that matter now. By combining best practices in participatory art with media technology’s innovations of language, form and process, the project is defining new approaches to co-creation and innovation, exploring novel audiovisual formats based in European cultural heritage, such as opera